Similarity of the cut score in test sets with different item amounts using the modified Angoff, modified Ebel, and Hofstee standard-setting methods for the Korean Medical Licensing Examination / 보건의료교육평가

Purpose@#The Korea Medical Licensing Exam (KMLE) typically contains a large number of items. The purpose of this study was to investigate whether there is a difference in the cut score between evaluating all items of the exam and evaluating only some items when conducting standard-setting. @*Methods@#We divided the item sets that appeared on 3 recent KMLEs for the past 3 years into 4 subsets of each year of 25% each based on their item content categories, discrimination index, and difficulty index. The entire panel of 15 members assessed all the items (360 items, 100%) of the year 2017. In split-half set 1, each item set contained 184 (51%) items of year 2018 and each set from split-half set 2 contained 182 (51%) items of the year 2019 using the same method. We used the modified Angoff, modified Ebel, and Hofstee methods in the standard-setting process. @*Results@#Less than a 1% cut score difference was observed when the same method was used to stratify item subsets containing 25%, 51%, or 100% of the entire set. When rating fewer items, higher rater reliability was observed. @*Conclusion@#When the entire item set was divided into equivalent subsets, assessing the exam using a portion of the item set (90 out of 360 items) yielded similar cut scores to those derived using the entire item set. There was a higher correlation between panelists’ individual assessments and the overall assessments.

Similarity of the cut score in test sets with different item amounts using the modified Angoff, modified Ebel, and Hofstee standard-setting methods for the Korean Medical Licensing Examination / 보건의료교육평가

Purpose@#The Korea Medical Licensing Exam (KMLE) typically contains a large number of items. The purpose of this study was to investigate whether there is a difference in the cut score between evaluating all items of the exam and evaluating only some items when conducting standard-setting. @*Methods@#We divided the item sets that appeared on 3 recent KMLEs for the past 3 years into 4 subsets of each year of 25% each based on their item content categories, discrimination index, and difficulty index. The entire panel of 15 members assessed all the items (360 items, 100%) of the year 2017. In split-half set 1, each item set contained 184 (51%) items of year 2018 and each set from split-half set 2 contained 182 (51%) items of the year 2019 using the same method. We used the modified Angoff, modified Ebel, and Hofstee methods in the standard-setting process. @*Results@#Less than a 1% cut score difference was observed when the same method was used to stratify item subsets containing 25%, 51%, or 100% of the entire set. When rating fewer items, higher rater reliability was observed. @*Conclusion@#When the entire item set was divided into equivalent subsets, assessing the exam using a portion of the item set (90 out of 360 items) yielded similar cut scores to those derived using the entire item set. There was a higher correlation between panelists’ individual assessments and the overall assessments.

Comparisons of item difficulty and passing scores by test equating in a basic medical education curriculum / 한국의학교육

PURPOSE@#Test equating studies in medical education have been conducted only for high-stake exams or to compare two tests given in a single course. Based on item response theory, we equated computer-based test (CBT) results from the basic medical education curriculum at the College of Medicine, the Catholic University of Korea and evaluated the validity of using fixed passing scores.@*METHODS@#We collected 232 CBTs (28,636 items) for 40 courses administered over a study period of 9 years. The final data used for test equating included 12 pairs of tests. After test equating, Wilcoxon rank-sum tests were utilized to identify changes in item difficulty between previous tests and subsequent tests. Then, we identified gaps between equated passing scores and actual passing scores in subsequent tests through an observed-score equating method.@*RESULTS@#The results of Wilcoxon rank-sum tests indicated that there were no significant differences in item difficulty distribution by year for seven pairs. In the other five pairs, however, the items were significantly more difficult in subsequent years than in previous years. Concerning the gaps between equated passing scores and actual passing scores, equated passing scores in 10 pairs were found to be lower than actual passing scores. In the other two pairs, equated passing scores were higher than actual passing scores.@*CONCLUSION@#Our results suggest that the item difficulty distributions of tests taught in the same course during successive terms can differ significantly. It may therefore be problematic to use fixed passing scores without considering this possibility.

Efficacies of Stem Cell Therapies for Functional Improvement of the β Cell in Patients with Diabetes: A Systematic Review of Controlled Clinical Trials

BACKGROUND AND OBJECTIVES: This study was performed to investigate whether stem cell therapy enhances β cell function by meta-analysis with proper consideration of variability of outcome measurements in controlled trial of type 1 diabetes mellitus (T1DM) and type 2 diabetes mellitus (T2DM) patients. METHODS: A systematic search was performed from inception to January 2018 in PubMed, EMBASE, and Cochrane databases. β cell function was assessed by stimulated C-peptide, fasting C-peptide, normal glycosylated hemoglobin levels (HbA1C), and exogenous insulin dose patterns. The quality of the studies were assessed by both the Cochrane Collaboration's Risk of Bias (ROB) for Randomized controlled trials and the Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I) for non-randomized controlled trials. RESULTS: From the selected final 15 articles, total of 16 trials were analyzed. There were 6 T1DM trials (total 153 cases) and 10 T2DM trials (total 457 cases). In T2DM patients, the changes in stimulated C-peptide, HbA1c, and exogenous insulin dose versus baseline showed a favorable pattern with a significant heterogeneity in stem cell therapy. In T1DM, there was no significant difference between control group and stem cell therapy group in three indicators except for HbA1c. Most of the studies were rated as having high risk of bias in the quality assessment. CONCLUSIONS: The stem cell therapy for DM patients is not effective in T1DM but seems to be effective in improving the β cell function in T2DM. However the observed effect should be interpreted with caution due to the significant heterogeneity and high risk of bias within the studies. Further verification through a rigorously designed study is warranted.

Adipose Tissue-Derived Stem Cell Therapy for Cavernous Nerve Injury-Induced Erectile Dysfunction in the Rat Model: A Systematic Review and Meta-Analysis Using Methodological Quality Assessment

BACKGROUND AND OBJECTIVES: Few studies were evaluated the effect of blindness on outcome in animal models, though a potential effect of blinding has been reported in clinical trials. We evaluated the effects of adipose tissue-derived stem cells (ADSCs) on cavernous nerve injury (CNI)-induced erectile dysfunction (ED) in the rat and examined how proper blinding of the outcome assessor affected treatment effect. METHODS AND RESULTS: We searched in Pubmed, EMBASE, Cochrane and Web of Science databases from inception to January 2019. We included CNI animal model, randomized controlled experiments, and ADSC intervention. Erectile function and structural changes were assessed by intracavernous pressure and mean arterial pressure (ICP/MAP) ratios, neuronal nitric oxide synthase (nNOS) levels, cavernous smooth muscle and collagen (CSM/collagen) ratios, and cyclic guanosine monophosphate (cGMP). RESULTS: Nineteen studies were included in the final meta-analysis. The ICP/MAP ratio of the ADSC treatment group increased compared to the control group (SMD=1.33, 95%CI: 1.11~1.56, I²=72%). The nNOS level (SMD=2.29, 95%CI: 1.74~2.84, I²=75%), CSM/collagen (SMD=2.57, 95%CI: 1.62~3.52; I²=85%), and cGMP (SMD=2.96, 95%CI: 1.82~4.10, I²=62%) were also increased in the ADSC treatment group. Preplanned subgroup analysis was conducted to explore the source of heterogeneity. Five studies with blinded outcome assessment were significantly less effective than the unblinded studies (SMD=1.33, 95%CI: 0.86~1.80; SMD=1.81, 95%CI: 1.17~2.46, respectively). CONCLUSIONS: ADSCs might be effective in improving erectile function and structural change in CNI-induced ED. However, non-blinded outcome assessors might cause detection bias and overestimate treatment efficacy. Therefore, the ADSC efficacy must be further evaluated with a rigorous study design to avoid bias.

Comparisons of item difficulty and passing scores by test equating in a basic medical education curriculum / 한국의학교육

PURPOSE: Test equating studies in medical education have been conducted only for high-stake exams or to compare two tests given in a single course. Based on item response theory, we equated computer-based test (CBT) results from the basic medical education curriculum at the College of Medicine, the Catholic University of Korea and evaluated the validity of using fixed passing scores. METHODS: We collected 232 CBTs (28,636 items) for 40 courses administered over a study period of 9 years. The final data used for test equating included 12 pairs of tests. After test equating, Wilcoxon rank-sum tests were utilized to identify changes in item difficulty between previous tests and subsequent tests. Then, we identified gaps between equated passing scores and actual passing scores in subsequent tests through an observed-score equating method. RESULTS: The results of Wilcoxon rank-sum tests indicated that there were no significant differences in item difficulty distribution by year for seven pairs. In the other five pairs, however, the items were significantly more difficult in subsequent years than in previous years. Concerning the gaps between equated passing scores and actual passing scores, equated passing scores in 10 pairs were found to be lower than actual passing scores. In the other two pairs, equated passing scores were higher than actual passing scores. CONCLUSION: Our results suggest that the item difficulty distributions of tests taught in the same course during successive terms can differ significantly. It may therefore be problematic to use fixed passing scores without considering this possibility.

Is Helicobacter pylori Associated Functional Dyspepsia Correlated With Dysbiosis?

BACKGROUND/AIMS: To assess the long-term effect of Helicobacter pylori eradication on symptomatic improvement according to the type of antibiotic and the duration of treatment in H. pylori-associated functional dyspepsia. METHODS: We searched Pubmed, Embase, CINAHL, and the Cochrane library databases for randomized controlled trials written in English and undertaken up to August 2016 that met our eligibility criteria. The search methodology used combinations of the following keywords: Helicobacter pylori OR H. pylori OR HP; dyspepsia OR functional dyspepsia OR non-ulcer dyspepsia; eradication OR cure OR treatment. The study outcome was the summary odds ratio (OR) for symptomatic improvement in H. pylori-associated functional dyspepsia with successful eradication therapy. Subgroup analyses were performed based on the type of antibiotic, and the duration of treatment, whether or not patients had symptoms of irritable bowel syndrome, and on race. RESULTS: Sixteen randomized controlled trials met the inclusion criteria. The summary OR for symptomatic improvement in patients in our eradication group was 1.33 (95% confidence interval [CI], 1.16–1.54; P < 0.01). In a subgroup analysis on type of antibiotic, symptomatic improvement with metronidazole-containing regimen (OR, 1.87; 95% CI, 1.26–2.77) was better than treatment with clarithromycin (OR, 1.29; 95% CI, 1.11–1.50). H. pylori eradication therapy given for 10–14 days was the more effective for symptom improvement than 7-day therapy. When the studies excluding irritable bowel syndrome cases were analyzed, there were no therapeutic effects of H. pylori eradication on symptomatic improvement. CONCLUSIONS: In the clinical setting, the most effective H. pylori eradication regimen for functional dyspepsia to provide relief of symptoms is a metronidazole-based treatment regimen for at least 10 days. The explanation for this is that H. pylori-associated functional dyspepsia could be associated with dysbiosis.

Analytical Case Study of Evaluation of Curriculum at a Medical School / 한국의학교육

PURPOSE: This study aims at to make intermediate-term evaluations of a curriculum by investigating its development, operation and outcomes 2 years after its revision. METHODS: A survey using 5-point-Lickert scale questionnaires was given to the group of directors who developed the curriculum, instructors who only used it in their classes, and a student group from the first and second grades. Focus group interviews were performed in the professor groups. RESULTS: Curricular reform was evaluated as being systematic, democratic, and positive in general. Both groups answered questions positively about the relevance of the integrated curriculum and introduction of clinical medicine (ICM), graded as 3.4 (professor) and 3.5 (student). As for problem-based learning (PBL) and the patient-doctor-society (PDS), the professor group responded more positively than students. The 'web-based learning center' was recognized positively by many more students (4.01) than professors (2.75). With regard to the education outcome, professors gave 3.3 points and students 3.5 to an item that asked 'whether students attained the learning goal or not?' Professors, through interviews, showed their satisfaction with the attempt to reform the curriculum, but they pointed out that long-term evaluations should be performed. CONCLUSION: The interim evaluation of the revised curriculum, from its planning to its effects, affirmed by several suggestions to be successful in the long run through 1) enhancement of systematic participation and communication, 2) further integration, 3) steady evaluation, 4) greater effort on professional development, and 5) active interaction between professors and students.